As methods of recording and transmitting video and sound information, encoding methods such as MPEG (Moving Picture Experts Group) have been established and MPEG-1, MPEG-2, and MPEG-4 specifications are available as international standard encoding methods. Also, specifications for improved coding efficiency, such as H.264/AVC (Advanced Video Coding), have been formulated. These methods are used as encoding methods in digital satellite broadcasting, DVDs, mobile phones and digital cameras.
On the other hand, in such applications, a video may be played back in a direction reverse to the order in which pictures were captured. As a means to produce a stream which can be played back in the reverse direction, a technique of achieving reverse playback using only intra-coded frames and bi-directional predicted frames is available (for example, JP-A No. H8-280024).
The above method has a problem that in decoding a transmitted stream on the receiving side, a large volume of memory for decoding is needed to play back a video from a desired point in the reverse direction. Let's suppose that a multi-viewpoint video captured by a multi-viewpoint object capturing system (301) as shown in FIGS. 3A and 3B is encoded by the conventional method. In this case, a picture at each viewpoint corresponds to a frame in a normal stream in the time direction. In FIGS. 3A and 3B, I represents an intra-coded frame, P a forward predicted frame, and B a bi-directional predicted frame. For example, if the picture at point B6 is to be played back, first P4 should be decoded (as derived) from I1, then B6 should be decoded from P4 and I7. This poses a problem that decoding of B6 requires a memory to process four pictures I1, P4, I7 and B6 and a remarkably larger volume of data must be processed. For this reason, a multi-viewpoint video transmission/display system which displays pictures while varying viewpoints in real time on the receiving side has not been realized. This problem occurs because the conventional video encoding method is designed to play back forward in the time direction but not to play back reversely or backward frame by frame.
The method described in the above Patent Document 1 also has a problem that forward predicted frames cannot be used and many intra-coded frames must be used to maintain the picture quality at a prescribed level or higher and a very large volume of data must be handled. Therefore, this method is not suitable for multi-viewpoint video transmission as mentioned above.
In view of the above conventional technique, the present invention provides a bit stream which can be played back reversely frame by frame with high picture quality and high coding efficiency, and a video encoder and a video decoder which cope with it. Also, the present invention is suitable for multi-viewpoint video encoding and provides a multi-viewpoint video bit stream, a video encoder and a video decoder which take advantage of this feature.